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(U)  Preface 

(U)  Between  May  2004  and  April  2005,  the  United  States  Air  Force  (USAF)  conducted 
an  intensive  assessment  of  the  biological  warfare  (BW)  threat  to  operations  at  Kunsan  Air 
Base  (AB),  Republic  of  Korea.  This  project,  known  as  the  Kunsan  Focused  Effort 
(KFE),  sought  to  develop  counter-biological  warfare  (C-BW)  strategies  and  procedures 
designed  to  mitigate  the  impacts  of  BW  attacks  on  Air  Force  operations  and  to  sustain 
and  recover  operations  in  a  broad  range  of  BW  environments.  The  KFE  initiative  was 
conducted  as  a  series  of  visits  by  a  cross-functional  team  of  subject  matter  experts 
representing  various  organizations,  e.g.,  the  Air  Staff  (A3SC),  Pacific  Air  Forces 
(PACAF),  Kunsan  AB  personnel,  Air  Force  Field  Operating  Agencies  (FOAs),  and 
supporting  contractor  personnel. 

(U)  In  support  of  this  effort,  a  series  of  seven  technical  reports  were  developed  to 
summarize  the  findings  of  the  KFE  analytical  efforts.  The  titles  of  these  technical  reports 
are  as  follows:  KFE  Threat  Analysis,  Residual  Hazard  Estimation  After  a  Biological 
Attack,  Aerosol  and  Surface  Sampling  and  Identification  Capability  Following  a 
Biological  Attack,  Biological  Background  Impact  on  Detector  Performance,  Analysis  of 
Masking  Criteria  with  Respect  to  BW  Trigger  Events,  Disease  Containment  Plan 
Analysis,  and  BW  Antibiotic  Treatment  Impact  on  Casualties  and  Sortie  Generation . 

(U)  It  is  important  to  note  that  the  seven  technical  reports  were  tied  to  the  objectives, 
guidelines,  and  timeframes  of  the  KFE  program.  Therefore,  there  are  some  inherent 
limitations,  as  well  as  areas  where  additional  work  and  analysis  is  required.  These  areas 
have  been  highlighted  in  their  respective  technical  report,  to  include  specific 
recommendations  for  further  analysis. 


UNCLASSIFIED 


1 


UNCLASSIFIED 


(U)  Abstract 

(U)  In  support  of  the  Kunsan  Focused  Effort  (KFE),  this  technical  report  describes  a 
project  that  examined  both  historic  and  contemporary  studies  related  to  background 
aerosol  materials  and  biological  weapon  (BW)  detection  systems.  The  goal  of  this 
project  was  to  summarize  how  current  BW  detection  systems  are  expected  to  behave  in  a 
non-laboratory  environment.  This  report  describes  past  monitoring  programs  and  their 
key  results.  The  programs  described  were  selected  because  their  data  results  were 
directly  applicable  to  the  background  problems  being  analyzed,  namely,  the  nature  of 
background  concentrations  and  fluctuations  of  aerosol  materials  that  are  known  to 
adversely  affect  BW  detection  systems.  Further,  a  modeling  effort  was  undertaken  to 
determine  if  it  was  possible  to  analytically  reproduce  aerosol  concentrations  observed  by 
aerosol  particle  counter  devices.  Input  data  was  obtained  from  a  large  number  of  Met- 1 
equipped  Portal  Shield  units.  Archived  data  sets  were  obtained  and  examined  for 
possible  use  as  input  for  statistical  analyses.  These  data  sets  were  subject  to  a  variety  of 
time  series  analyses  to  determine  if  descriptive  statistics  could  be  obtained  and  used  to 
produce  a  statistically  similar  concentration  history.  An  analysis  of  these  data  found  that 
simple  particle  counters  designed  to  monitor  background  concentration,  such  as  the  Met- 
1  used  in  the  Portal  Shield,  are  subject  to  a  large  number  of  trigger  events.  Particle 
monitor  types  of  triggers  all  suffer  from  background  concentration  fluctuations,  which  are 
frequent,  natural  and  man-made,  and  unpredictable.  Each  unique  type  of  particle  monitor 
requires  a  field  study  to  detennine  how  its  particular  suite  of  sensors  will  react  to  the 
background  materials  at  different  locations.  Each  new  detection  technology  requires  a 
significant  field  study  effort  to  detennine  how  the  new  technology  “sees”  the 
environment. 
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(U)  Summary 

“A  knowledge  of  the  background  atmosphere  that  may  be 
experienced  by  the  detection  system  is  important  for  establishing  the 
limitations  and  suitable  operating  conditions  for  any  sensor.” 

—  Final  Report  on  the  Research  Program  on  BW 
Detection,  Space  General,  AD480357,  1966. 

(U)  The  goal  of  this  project  was  to  examine  the  effects  of  background  aerosol  materials 
on  the  operation  of  biological  agent  detection  systems  currently  in  use  on  United  States 
Air  Force  (USAF)  facilities.  The  effort  was  initiated  as  part  of  the  Kunsan  Focused 
Effort  (KFE),  a  project  designed  to  provide  a  foundation  for  the  compilation  of  a  USAF 
counter-biological  warfare  (C-BW)  concept  of  operations  (CONOPs).  Knowledge  of 
BW  detection  systems  is  an  important  component  of  the  C-BW  CONOPs.  Statements, 
such  as  the  one  made  in  the  1966  Space  General  report,  indicate  that  background  aerosol 
materials  have  affected  and  continue  to  affect  the  operation  of  these  detection  systems. 
This  project  sought  to  examine  both  historic  and  current  study  projects  related  to 
background  aerosol  materials  and  BW  detection  systems,  and  to  produce  a  set  of 
guidelines  and  conclusions  on  how  current  BW  detection  systems  are  expected  to  behave 
in  a  non-laboratory  environment.  The  desire  was  to  construct  a  C-BW  CONOPS  that 
would  take  advantage  of  knowledge  concerning  the  actual  perfonnance  of  BW  detection 
systems.  It  is  well  known  that  BW  detection  in  general  is  a  difficult  problem  and  that 
each  type  of  detection  system  has  strengths  and  weaknesses.  Knowing  when  a  BW 
detection  system  is  likely  to  work  or  not  work,  (specifically  as  affected  by  background 
material),  allows  other  defensive  procedures  to  be  implemented,  (e.g.,  increased 
protection,  medical  prophylaxis  and  treatments,  etc.),  thus  providing  the  most  robust 
defensive  capability,  while  minimizing  vulnerability  to  BW  challenges. 
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(U)  Introduction 

(U)  The  Nature  of  Background  Aerosols 

(U)  Atmospheric  background  aerosols  have  been  studied  for  many  years  due  to  their 
adverse  effects  on  humans  and  the  environment.  Fine  particles  consist  of  materials  that 
act  as  aeroallergens,  contribution  to  acid  deposition  (rain),  and  reduce  visibility. 
Background  aerosols  also  affect  the  performance  of  biological  agent  detection  systems 
that  rely  on  the  capture  and  identification  of  aerosol-delivered  agent  material. 

(U)  Atmospheric  Aerosols  (AAs)  are  defined  as  particles  with  aerodynamic  particle  sizes 
of  25  microns  or  less.  AAs  are  characterized  by  their  source,  composition,  size,  and 
concentration.  Sources  of  AA  include  natural  flora,  vapor  condensates  (both  natural  and 
manmade),  and  erosion-produced  dust.  The  composition  of  AAs  varies  widely  and 
includes  pollen,  fungal  spores,  bacteria  and  bacterial  spores,  sea  spray  residues,  soot, 
dust,  tire  rubber,  condensation  nuclei,  and  wind -produced  erosion  particles.  While 
aerosol  particle  sizes  range  from  25  microns  to  less  than  0.1  microns,  this  report 
addresses  particles  in  the  10  to  0.5  micron  range.  These  particle  sizes  are  typical  of  those 
produced  during  the  dissemination  of  biological  agent  materials  designed  to  travel  for 
long  downwind  distances  (up  to  100  kilometers).  The  ability  of  aerosol  particles  in  the 
0.5  to  10  micron  range  to  adhere  to  deep  lung  surfaces  is  one  reason  that  the 
Environmental  Protection  Agency  (EPA)  maintains  an  aerosol  monitoring  program.  For 
example,  particles  larger  than  10  microns  are  considered  aerosols,  but  their  larger  size 
and  fall  velocities  limit  their  aerosol  travel  distance.  Particles  smaller  than  0.5  microns 
are  ignored  due  to  their  limited  ability  to  adhere  to  deep  lung  surfaces1.  Moreover,  many 
agents,  bacterial  agents  especially,  cannot  be  delivered  in  sub-micron  particle  sizes  due  to 
the  size  of  the  organism  or  spore. 

(U)  Scientists  have  known  for  many  years  that  background  atmospheric  particles 
characteristics  include:  1)  the  particles  are  of  the  inhalable  size  (.5-10  microns);  2)  the 
particles  can  fluoresce  (41%  of  the  two  to  five  micron  size)  ;  3)  they  contain  bacteria  and 
spores3,  and  4)  the  particle  concentrations  vary  by  location  and  time. 

(U)  Aerosol  concentrations  vary  widely  by  location,  by  time,  and  by  composition.  Many 
aerosol  monitoring  programs  have  documented  these  concentration  changes  as  part  of 
their  effort  to  characterize  specific  background  properties.  The  aerosol  concentration 
fluctuations  over  time  are  important  to  the  biological  agent  detection  problem,  as  many 
detector  designs  are  based  on  detecting  the  increase  or  decrease  in  concentrations  that  are 
characteristic  of  the  passing  aerosol  clouds.  A  key  assumption  used  in  many  detection 
systems  is  that  the  concentration  change  over  time  from  a  passing  artificially  generated 
aerosol  cloud  is  distinctly  different  than  most  natural  concentration  variations  both  in 
time  scale,  (i.e.,  time  between  initial  increase  and  final  decrease),  and  in  magnitude  of  the 
aerosol  concentration. 
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(U)  Background  Aerosols  and  Biological  Detection  Systems 

(U)  Biological  aerosol  detection  systems  can  be  affected  by  background  aerosols  in  two 
ways.  First,  the  background  aerosols  affect  systems  designed  to  monitor  aerosol 
concentrations  in  near  real  time  in  order  to  identify  a  passing  aerosol  cloud.  Detection 
algorithms  are  designed  to  monitor  the  ambient  aerosol  concentration  and  to  detect  the 
increase  and  decrease  concentration  pattern  of  a  passing  cloud.  Simple  concentration 
monitors  examine  particle  counts  in  one  or  more  size  categories.  These  devices  count  the 
number  of  particles  in  an  air  sample.  By  knowing  the  volume  of  the  air  sample  and  the 
number  of  particles  it  contains,  particle  concentration  is  easily  computed.  More  elaborate 
systems  examine  the  characteristics  of  these  particles,  such  as  their  fluorescence  or 
reflectivity,  when  they  are  illuminated  by  an  artificial  light  source.  Such  efforts  are 
designed  to  differentiate  between  simple  non-organic  and  organic  compounds.  The  idea 
behind  these  systems  is  that  organic  material  (including  BW  agent)  is  a  fraction  of  the 
overall  airborne  aerosol  material,  thus  a  large  portion  of  the  background  material  is 
ignored.  Even  more  elaborate  particle  concentration  monitors  are  designed  to 
differentiate  between  different  classes  of  organic  compounds.  The  differentiation 
between  diesel  exhaust  and  other  organic  materials  is  one  goal  of  these  systems. 

(U)  Second,  the  background  aerosol  is  a  concern  for  the  identification  process.  Many 
detection  systems  collect  an  aerosol  sample,  concentrate  it,  and  then  submit  it  for 
identification  to  some  sort  of  an  assay  device.  The  difficulty  here  is  being  able  to  identify 
an  agent  particle  from  the  other  background  material  collected  in  the  sample.  Various 
monitoring  programs  have  been  performed  as  a  means  to  identify  the  range  of  interfering 
materials  collected  and  their  concentrations  so  that  optimal  identification  assays  can  be 
designed. 

(U)  In  summary,  background  aerosol  concentrations  affect  biological  detection  systems 
two  ways.  First,  they  detennine  the  background  concentration  and  concentration 
fluctuations  monitored  by  near  real  time  particle  concentration  monitors.  These  systems 
are  designed  to  provide  a  fast  acting  trigger  signal  to  either  an  alann  or  a  sample 
collection  device.  Second,  they  determine  the  mix  and  amount  of  background  material 
collected  and  submitted  to  an  identification  assay.  The  identification  assay  must 
determine  the  presence  or  absence  of  agent  signal  from  the  possible  interfering  signal 
originating  from  the  collected  background  material. 

(U)  Monitoring  Programs  and  Key  Results 

(U)  This  section  provides  a  selected  summary  of  past  monitoring  programs  and  their  key 
results.  There  have  been  many  programs  performed  worldwide  covering  a  wide  range  of 
goals.  The  programs  mentioned  here  are  selected  for  their  data  results  that  are  directly 
applicable  the  background  problems  described  above,  namely,  the  nature  of  background 
concentrations  and  fluctuations  of  aerosol  materials  that  are  known  to  adversely  affect 
BW  detection  systems,  particularly  in  the  two  ways  described  above. 

(U)  M18  Detector  Program  and  Interferents  Using  Fluorescence 

(U)  In  the  late  1970s,  the  US  Army  was  developing  an  automated  biological  detection 

system  that  relied  on  detection  of  chemiluminescence  resulting  from  a  chemical  reaction 
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with  luminal  and  porphyrin4.  During  this  time,  other  research  programs  were  conducted 
to  determine  the  characteristics  of  aerosol  particles  that  were  commonly  found  in  the 
atmosphere  and  their  characteristic  size.  One  program  reported  that  almost  all  pollen 
particles  were  relatively  large,  (i.e.,  >20  microns)  and  that  fungal  spores  were  almost  all 
around  five  microns  in  size5.  This  meant  that  it  was  an  easy  process  to  exclude  pollen 
from  collected  aerosol  materials.  Additionally,  this  report  noted  a  long  list  of  common 
materials  that  fluoresced,  including  materials  of  biological  origin  (both  natural  and 
manmade),  manmade  fibers,  and  a  large  number  of  both  organic  and  inorganic 
compounds.  These  data  implied  that  the  use  of  simple  fluorescent  techniques  could  not 
be  relied  on  to  filter  out  non-organic  or  non-biological  materials6. 

(U)  Miscellaneous  References  and  Research  Results 
(U)  Additional  research  efforts  reported  the  following  results: 

•  (U)  Forty-one  percent  of  the  particles  in  the  two  to  five  micron  size  range 
fluoresced7. 

8  9 

•  (U)  Various  types  of  bacteria  were  commonly  found  in/with  aerosol  particles  ’  . 

•  (U)  Indoor  mineral  particles,  such  as  silicate,  salt,  and  others  in  the  three  to  five 
micron  size  range  were  commonly  found  in  indoor  air  samples10. 

•  (U)  Time-resolved  concentration  profiles  of  bacteria  and  spores  showed  that  spore 
concentrations  of  11,000  per  cubic  meter  (mA3)  occurred  and  that  the 
concentration  curve  contained  many  “bumps”  resembling  an  aerosol  cloud 
passage,  as  well  as  many  short  duration  spikes11. 

•  (U)  A  large  number  of  fungi  species  commonly  found  in  the  atmosphere  (in 
Puerto  Rico)  had  concentrations  that  could  reach  250,000  spores/mA3,  but  that  the 
common  concentration  was  a  few  thousand  per  mA3.  The  report  also  contains  an 
estimate  that  a  human  inhales  between  80,000  and  100,000  spores  per  day  and 
that  there  were  more  airborne  fungi  during  the  day  than  during  the  night  . 

•  (U)  Regardless  of  altitude,  molds  constituted  70  percent  of  the  total  airborne 
micro  flora;  bacteria  constituted  between  19  and  26  percent;  and  yeast  and 
actinomycetes  filled  the  remaining  percents.  The  report  also  noted,  “a  significant 
portion  of  the  viable  microorganisms  in  the  air  were  in  the  particle  size  range  of 
three  to  five  microns.”  and  that  airborne  bacteria  concentrations  increased  from 
283/mA3  to  17,900  /mA3  downwind  from  an  activated  sludge  sewage  treatment 
unit13. 

•  (U)  Many  other  references  indicated  the  common  presence  of  airborne  molds, 
fungi,  and  bacteria  particles  all  over  the  world  in  similar 

„  14, 15, 16, 17, 18,1 9,20 , 2 1 

quantities  . 

•  (U)  Of  the  particle,  bacteria,  and  fungal  spore  concentrations  measured  inside 
homes  and  daycare  centers,  bacteria  concentrations  of  up  to  8500  colony  forming 
units  (cfu)/mA3  were  observed.  Fungal  spore  concentrations  of  up  to  5620 
cfu/mA3  were  reported22.  As  these  data  were  reported  in  cfu/mA3,  higher  airborne 
concentrations  of  non-reproducing  organisms  were  certainly  present.  The  report 
states  that  there  were  approximately  50  total  airborne  bacteria  for  every  culturable 
bacterium23. 


UNCLASSIFIED 


9 


UNCLASSIFIED 


•  (U)  One  report  included  a  wide  list  of  airborne  spore  types  and  concentration 
statistics  for  various  locations,  to  include  many  entries  for  concentrations  above 
100,000  spores/mA324. 

•  (U)  (The  aerosol)  “Background  is  variable  and  unpredictable”  and  “this  will 

25 

impact  detector  response" 

(U)  Improved  Biological  Agent  Detection  Systems  (IBADS)  and  Portal  Shield  Unit 
Results  from  Overseas  Locations 

(U)  One  report  concerned  hourly  samples  collected  using  the  IBADS  and  Portal  Shield 
units  at  Osan  Air  Base  (AB)  and  Kunsan  AB  in  the  Republic  of  Korea,  in  Kuwait,  and 
Camp  Doha,  Bahrain.  The  samples  were  analyzed  using  polymerase  chain  reaction 
(PCR)  and  gene  sequencing  to  identify  background  organisms  and  to  specifically  look  for 
the  biological  agents  Bacillus  anthracis  (B.  anthracis),  Francisella  tularensis  (F. 
tularensis),  and  Yersinia  pestis  (Y.  pestis).  The  report  did  not  provide  details  on  the 
sample  collection,  but  the  Portal  Shield  unit  collected  for  a  five-minute  duration  drawing 
air  at  close  to  one  mA3/min  air  flow  rate.  This  is  a  relatively  short  duration  sample  as 
compared  to  samples  collected  by  the  dry  filter  unit  (DFU)  type  devices,  which  collect 
samples  for  hours.  The  results  were  that  zero  out  of  989  samples  from  Osan  AB  were 
positive  for  the  three  agents  listed  above;  one  out  of  565  samples  from  Kunsan  AB  was 
positive  for  F.  tularensis;  one  out  of  519  samples  from  Kuwait  was  positive  for  F. 
tularensis',  and  one  out  of  516  samples  from  Camp  Doha,  Bahrain  was  positive  for  both 
F.  tularensis  and  Y.  pestis.  These  data  suggest  that  about  one  out  of  every  500  samples 
assayed  using  sensitive  identification  techniques,  (such  as  deoxyribonucleic  acid  [DNA] 
amplification),  could  be  expected  to  produce  a  positive  result  for  an  endemic  biological 
agent  organism.  The  authors  of  this  report  indicated  that  the  amount  of  agent  material  in 
these  samples  was  small,  and  thus,  their  results  were  from  endemic  organisms  rather  than 
deliberately  released  agents.  They  did  not  provide  material  to  indicate  how  they  would 
detennine  what  level  was  sufficient  to  indicate  an  attack,  nor  how  their  techniques  could 
quantify  these  data.  These  data  suggest  the  need  for  an  identification  technique  that  is 
both  sensitive  and  able  to  quantify  the  amount  of  agent  material  present  in  a  sample26. 

(U)  Defense  Advanced  Research  Projects  Agency  (DARPA)  Biological  Agent  Warning 
Sensor  (BA  WSJ  and  Met-1  Results 

(U)  “A  strong  mathematical  correlation  between  data  from  an  ultraviolet  aerodynamic 
particle  sizer  (UVAPS)  and  simulant  concentration  was  not  found.”  This  result  indicated 
that  using  relatively  simple  ultraviolet  (UV)  particle  concentration  monitoring  was  not 
useful  as  a  trigger  device  because  the  UV  signal  did  not  correlate  well  with  the 

97 

agent/simulant  concentration  signal"  . 

(U)  Background  Particle  Concentrations  Around  Military  Activities 
(U)  Aerosol  measurements  were  made  during  a  series  of  trials  at  Fort  Sill,  Oklahoma 
involving  truck  and  tank  movements,  muzzle  blasts,  and  sub-munition  bursts"  .  These 
trials  were  designed  to  determine  the  obscuring  effects  from  these  activities  in  both  the 
visual  and  infrared  spectra.  The  key  findings  were  that  the  dust  particles  raised  by  these 
activities  had  number  median  diameters  (NMDs)  in  the  one  to  four  micron  range. 
Additionally,  aerosol  dosage  levels  were  0.5  g  min/mA3  (peak)  for  tracked  vehicle 
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operations  and  about  an  order  of  magnitude  less  for  muzzle  blasts  and  sub-munition 
explosions.  Using  an  assumption  of  a  particle  size  of  three  microns,  a  material  density  of 
2g/cmA3,  and  the  observation  that  the  cloud  passage  time  was  about  two  minutes,  results 
in  an  estimation  of  aerosol  particle  concentrations  near  1.0  x  1 0 1 0  ( 1  e  1 0)  particles/mA3  for 
the  tracked  vehicle  activities  and  le9  particles/mA3  for  the  muzzle  blast  and  sub-munition 
explosions.  These  are  huge  concentrations  and  even  if  less  intensive  activities  were 
involved,  it  is  clear  that  aerosol  detection  systems  will  have  to  deal  with  a  large  number 
of  interfering  particles  in  the  vicinity  of  military  ground  operations. 

(U)  Particle  Counters  Used  as  Trigger  Devices 

(U)  This  section  provides  a  brief  description  of  current  systems  that  monitor  aerosol 
particle  concentrations  in  order  to  provide  a  relatively  fast  trigger  signal  to  an  aerosol 
collection  system. 

(U)  Met-1  Used  in  Portal  Shield 

(U)  The  Portal  Shield  system  uses  individual  Port  Shield  units  each  containing  a  Met-1 
real-time  particle  counter.  The  Met-1  counts  the  particles  that  are  present  in  an  air  stream 
in  six  size  categories.  The  device  illuminates  air  flowing  through  a  tube  with  a  laser  and 
monitors  light  reflected  by  aerosol  particles  contained  in  the  air  stream.  Properties  of  the 
reflected  light  allow  the  device  to  determine  the  size  of  the  particle  being  illuminated. 
The  particle  counts  are  reported  every  nine  seconds  to  a  trigger  algorithm.  The  trigger 
algorithm"  maintains  a  moving  average  concentration  level  to  account  for  natural  slow 
variations  in  background  particle  concentrations.  The  algorithm  looks  for  relatively  fast 
rising  concentration  values  that  are  characteristics  of  discrete  aerosol  clouds,  such  as 
would  be  generated  by  a  remote  aerosol  generator.  If  the  algorithm  detects  a  possible 
aerosol  cloud,  it  triggers  an  aerosol  collector/concentrator  to  collect  a  sample  of  the 
passing  aerosol  cloud.  The  Portal  Shield  units  are  arranged  as  an  arrayed  system  in  order 
to  minimize  the  possibility  of  a  small  cloud  passing  the  area  without  detection  and  to 
allow  multiple  units  to  correlate  a  cloud.  The  system  algorithm  looks  at  multiple  Portal 
Shield  units  to  see  if  they  trigger  in  time  and  space  consistent  with  a  passing  discrete 
cloud,  based  on  wind  direction  and  wind  speed.  If  correlated  units  trigger,  then  the 
collected  samples  from  these  units  are  submitted  for  immunoassay  identification,  under 
the  assumption  that  multiple  units  have  detected  an  aerosol  cloud  and  that  the  aerosol 
cloud  might  be  a  biological  agent  cloud.  Both  the  trigger  algorithm  for  individual  units 
and  the  system  correlation  algorithm  have  been  tuned,  using  data  collected  over  a  wide 
variety  of  locations  and  for  long  durations  to  minimize  the  number  of  samples  collected 
and  submitted  to  the  identification  assay.  That  is  not  to  say  that  the  algorithm  has  been 
tuned  to  minimize  the  number  of  aerosol  clouds  detected.  Experience  has  shown  that  the 
system  does  a  good  job  of  detecting  (triggering  on)  aerosol  clouds.  Trigger  events  have 
been  observed  to  occur  in  large  numbers,  especially  when  vehicle  traffic  is  occurring  and 
during  daytime  operation. 

(U)  Biological  Agent  Warning  System  (BA  WSJ 

(U)  BAWS  is  an  advanced  particle  monitor  developed  by  Lincoln  Laboratories  to  supply 
the  particle  monitoring  function  for  the  Joint  Biological  Point  Detection  System  (JBPDS) 
and  to  replace  or  augment  the  Met-1  device  in  the  currently  fielded  Portal  Shield  systems. 
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Like  the  Met-1,  BAWS  illuminates  the  air  flowing  in  a  tube  with  a  laser  (UV  laser  in  this 
case),  and  monitors  three  spectral  signals  returned  by  entrained  particles.  These  signals 
indicate  the  relative  concentration  of  the  aerosol,  (but  not  the  actual  particle  count),  and 
allow  the  illuminated  material  concentration  estimates  to  be  placed  in  material  category 
bins.  The  idea  behind  the  device  is  that  category  bin(s)  associated  with  background 
interferents  are  different  from  the  bins  that  contain  agent  materials,  (i.e.  they  look 
different  in  the  three  different  spectral  bands).  A  large  monitoring  program  was 
performed  to  determine  the  spectral  returns  of  normal  background  materials  and  their  bin 
characteristics.  Likewise,  laboratory  tests  were  undertaken  to  determine  the  spectral 
signals  returned  by  agent  materials  to  establish  their  bin  characteristics.  The  BAWS  unit 
contains  a  trigger  algorithm  based  on  these  data  and  the  moving  average  of  the 
background  concentration.  The  goal  of  the  BAWS  is  to  eliminate  many  of  the  trigger 
events  produced  by  the  more  simple  Met-1  that  are  caused  by  natural  or  manmade 
background  aerosol  clouds  that  do  not  contain  agent  (or  possible  agent).  As  with  the 
Met-1 /Portal  Shield  program,  a  large  BAWS  monitoring  program  was  undertaken  to 
characterize  the  BAWS  background  signals  and  to  tune  the  trigger  algorithm. 

(U)  Continuous  Sample  Collection  Devices 

(U)  Rather  than  rely  on  a  trigger  device  to  turn  on  an  aerosol  collector  and  then  initiate  a 
possible  identification  process,  a  second  type  of  point  biological  agent  collector,  called  a 
continuous  sample  collector,  is  in  common  use.  These  devices  draw  a  continuous  air 
stream  through  an  inlet  (designed  to  separate  out  large  particles),  concentrate  the  particles 
of  the  desired  size  range,  and  collect  these  particles  either  on  a  filter  medium  or  in  a 
liquid.  The  collected  sample  is  transported  to  a  laboratory  for  sample  processing  and 
analysis.  Because  these  devices  are  continuously  collecting  aerosol  material,  they  are  not 
susceptible  to  missing  an  agent  cloud  due  to  the  failure  of  a  trigger  device.  These  devices 
typically  collect  aerosol  samples  with  collection  durations  ranging  from  one  to  24  hours. 
The  shorter  sample  intervals  result  in  the  ability  to  detect  a  possible  attack  quicker  than 
the  longer  sampling  intervals,  assuming  the  individual  samples  are  processed  promptly. 
Disadvantages  to  this  approach  are  as  follows:  1)  each  sample  must  be  processed;  2)  there 
is  a  significant  time  delay  between  sample  collection  and  sample  processing  results;  3) 
sample  material  remains  on/in  the  collection  media  for  a  long  period  of  time,  which  may 
adversely  affect  the  sample  processing  step,  (e.g.,  vegetative  bacterial  cells  cannot  be 
cultured  after  drying  out  on  filter  media);  and  4)  large  amounts  of  background  material 
are  included  in  the  sample. 

(U)  Indoor  Applications 

(U)  DARPA  undertook  a  program  to  examine  the  ability  of  particle  counters  and  BAWS 
to  ignore  background  aerosol  fluctuations  that  normally  occur  in  a  variety  of  occupied 
buildings.  Their  results  showed  that  particle  counters  could  only  operate  at  a  reasonable 
false  alarm  rate  if  their  sensitivity  was  adjusted  to  2000  -  3000  particles  per  liter  (PPL) 
background  concentrations.  The  BAWS  performed  better  requiring  a  sensitivity  of  300 
PPL.  The  trigger  events  observed  at  these  sensitivities  were  often,  but  not  always 
associated  with  cleaning  activities. 
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(U)  Methods 


(U)  Modeling  Background  Aerosol  Concentrations 

(U)  A  modeling  effort  was  undertaken  to  determine  if  it  was  possible  to  analytically 
reproduce  aerosol  concentrations  observed  by  aerosol  particle  counter  devices.  Input  data 
was  obtained  from  a  large  number  of  Met- 1  equipped  Portal  Shield  units.  Of  note,  Met-1 
measurements  and  other  Portal  Shield  data  are  routinely  archived,  which  resulted  in  a 
plethora  of  data.  A  number  of  these  archived  data  sets  were  obtained  and  examined  for 
possible  use  as  input  for  statistical  analyses.  Examination  of  the  raw  data  indicated  that 
manual  processing  was  required  due  to  time  gaps  and  other  obvious  data  anomalies 
(reference  Figure  1). 
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(U)  Figure  1:  Osan  AB  Particles  Per  Cubic  Centimeter  (PPCC)  Data  from  April/May  1999.  This 
dataset  contains  obvious  discontinuities.  Three  hundred  and  fifty  three  datasets  were  edited  so  only 
continuous  sampling  data  were  analyzed. 

(U)  Of  the  data  sets  obtained,  only  353  passed  the  acceptance  criteria  of  being  continuous 
in  time  and  having  a  duration  period  of  at  least  six  hours.  Figure  2  is  an  example  of  one 
of  these  data  sets. 
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(U)  Figure  2:  Log  10  PPCC  at  Kunsan  AB,  4  -  9  May  2000.  This  is  an  example  of  an  accepted  dataset 
containing  no  time  gaps. 


(U)  These  data  sets  were  subject  to  a  variety  of  time  series  analyses  to  determine  if 
descriptive  statistics  could  be  obtained  and  used  to  produce  a  statistically  similar 
concentration  history.  One  approach  was  to  compute  the  point-to-point  difference  in 
particles  per  cubic  centimeter  (PPCC),  the  data  value  reported  by  the  Met-1,  for  a 
particular  size  channel.  The  top  curve  of  Figure  3  shows  an  example  computed  from  the 
Figure  2  data.  The  difference  data  was  then  fitted  with  either  a  standard  probability 
distribution  functions  (PDF)  using  a  statistical  analysis  program  (Figure  4),  or  an 
empirical  PDF  (bottom  of  Figure  3)  that  was  computed  directly  from  the  data. 
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(U)  Figure  3:  Initial  Difference  Data  and  PDF  of  Difference  Data.  This  chart  is  a  point-to-point 
difference  in  PPCC  computed  from  the  Figure  2  data  (Kunsan  AB,  4-9  May  2000).  The  extreme  value 
left  is  -1.9  and  the  extreme  value  right  is  2.7.  There  are  100  uniformly  spaced  bins  and  the  mean  of 
distribution  is  0. 
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(U)  Figure  4:  Frequency  Distribution  Fit  to  Point-to-Point  Difference  (PPCC)  Kunsan  AB,  4-9  May 
2000  (Met-1  data).  This  chart  is  the  difference  data  fitted  with  an  empirical  PDF  computed  directly  from 
the  data. 
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(U)  A  simulated  difference  series  is  then  produced  by  random  sampling  from  the  PDF  or 
by  using  a  moving  average  method  combined  with  random  sampling  from  the  PDF.  The 
bottom  of  Figure  5  shows  the  results  of  such  a  process  using  statistical  data  computed 
from  the  curve  shown  in  the  top  portion  of  the  figure.  Although  this  simulation  of  the 
original  difference  data  looks  similar,  the  process  of  reconstruction  of  the  simulated 
signal  (from  the  difference  data)  produces  results  that  are  obviously  different  as  shown  in 
Figure  6.  The  downward  pointed  spikes  in  the  simulated  signal  are  not  present  in  the 
original  signal  and  suggest  that  some  sort  of  time  correlation  is  needed  with  the 
difference  data  to  ensure  that  upward  spikes  are  followed  at  some  time  by  a  downward 
spike  of  similar  magnitude.  Various  types  of  moving  average  (MA)  and  conditional 
probability  schemes  were  attempted  to  rectify  this  problem. 


UNCLASSIFIED 


4 
3 
2 
1 
D 
-1 
-2 
-3 
-4  1 


Input  Data  (difference  time  sequence) 


100  200  300  400  500  600  700  800  900  1000  1100 

Prediction  from  Observed  Cumulative  Distribution  Functions  (CDF) 


900  1000 


100  200  300  400  500  600  700  800 

With  a  Moving  Average  (MA)  (1)  Model,  Theta  (1)  =  .7732 

(U)  Figure  5:  Reconstruction  of  the  Simulated  Signal  (PPCC  Difference  Data)  Using  a  Moving 
Average  Model  and  Observed  Cumulative  Distribution  Statistics. 


(U)  The  moving  average  techniques  were  able  to  reproduce  the  difference  data  well 
(reference  Figure  5),  however,  they  failed  when  concentration  features,  (such  as  an 
increase  in  concentration  caused  by  a  passing  aerosol  cloud),  were  present  having  time 
scales  different  from  the  moving  average  scale.  The  conditional  probability  techniques 
failed  by  not  having  enough  memory,  which  allowed  the  simulated  signal  to  “wander” 
(reference  Figures  6  and  7). 
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(U)  Figure  6:  Reconstructed  Time  Series  from  Original  and  Predicted  Difference  Data  Using  a 
Conditional  Probability  Technique.  Y  scales  offset  for  clarity. 
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(U)  Figure  7:  Reconstructed  Time  Series  Based  on  Conditional  Probability  Approach  -  No  Moving 
Averages.  Y  scales  offset  for  clarity. 
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30 

(U)  The  best  results  were  obtained  using  an  auto-regression  technique'  .  This  process, 
called  Autoregressive  to  Anything  (ARTA),  combines  auto-regression  statistics  for  the 
time  series  data  with  a  probability  distribution  to  generate  a  simulated  time  series  that  has 
the  correct  auto-correlation  and  cumulative  statistics.  Figure  8  shows  the  results  of  this 
technique  on  a  short  input  signal.  This  technique  can  generate  realistic  simulated 
background  concentration  time  series  data,  but  these  data  contain  no  more  “information” 
than  the  initial  data  sets  used  to  compute  the  auto-correlation  coefficients  and  the  range  of 
values  contained  in  the  PDF.  The  ARTA  process  contains  a  wide  range  of  available 
PDFs,  so  the  realism  of  the  simulated  time  series  depends  on  how  well  the  chosen  PDF 
matches  the  characteristics  of  the  real  background  data.  The  conclusion  for  this  modeling 
effort  was  that  there  are  time  series  simulation  techniques  that  can  generate  time  series 
data  that  may  be  useful  for  some  simulation  and  modeling  applications.  However,  these 
generated  data  sets  cannot  contain  any  more  information  about  the  background 
concentration  than  the  real  concentration  time  series  data  used  to  generate  the  simulation 
generation  statistics.  Furthermore,  this  modeling  effort  did  not  attempt  to  replicate  any  of 
the  spatial  correlation.  This  additional  level  of  complexity  would  be  necessary  for 
simulating  the  operation  of  detection  arrays  that  utilize  multi-sensor  spatial  and  temporal 
correlations  associated  with  cloud  traversal  across  detection  arrays  utilizing  complex 
multi-detector  algorithms. 
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(U)  Figure  8:  Autoregressive  to  Anything  (ARTA)  Technique  Using  Short  Input  Signal. 
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(U)  Met-1  Performance 

(U)  Data  used  to  generate  artificial  background  aerosol  profiles  were  also  used  to  analyze 
the  Met-1  performance  with  respect  to  false  triggers.  The  353  background  profiles 
containing  over  2000  hours  of  Met-1  samples  were  submitted  to  the  Portal  Shield  trigger 
algorithm  to  determine  the  number  of  trigger  events  expected  over  a  period  of  time. 
Details  of  this  Portal  Shield/Met- 1  trigger  analysis  are  in  Appendix  A.  The  results  of  this 
analysis  (reference  Figure  9)  indicated  that  the  mean  number  of  samples  per  day  per  Met- 
1  is  about  19  (median  of  15).  There  were  some  instances  where  the  background 
concentration  fluctuations  were  so  common  that  a  Met-1  would  be  expected  to  trigger 
over  100  times  per  day  (essentially  continuously).  Because  of  the  design  of  the  trigger 
algorithm,  each  trigger  event  is  caused  by  a  concentration  time  profile  that  resembles  the 
passage  of  an  aerosol  cloud.  Because  the  Met- 1  device  is  a  mature  sampling  device,  it  is 
likely  that  these  trigger  events  were  caused  by  aerosol  clouds  (and  not  by  spurious  Met-1 
operating  anomalies).  If  these  trigger  events  were  real  aerosol  clouds,  then  there  is  every 
reason  to  assume  that  these  clouds  would  constitute  a  system  trigger  for  the  Portal  Shield 
array  operating  in  smart  mode.  In  other  words,  these  clouds,  assuming  they  were  large 
enough  to  cover  multiple  Portal  Shield  units,  would  be  correlated  with  wind  direction  and 
wind  speed  so  that  the  Portal  Shield  system  would  indicate  that  the  cloud  was  a  possible 
biological  aerosol  cloud  and  cause  the  samples  collected  by  the  triggered  units  to  be 
submitted  for  immunoassay  identification.  The  conclusion  is  that  the  Portal  Shield 
system  operating  in  smart  mode  would  be  expected  to  trigger,  (i.e.,  initiate  immunoassay 
identification  at  the  triggered  units)  close  to  19  times  per  day  (average)  based  on  the 
collected  data  analyzed  here. 


UNCLASSIFIED 


Samples  Per  Day 
(353  Background  Profiles) 

Histogram 


60- 


c 

0) 

3 

CT 

2  40- 


Most  Profiles  Resulted  in 
Less  Than  Five  Triggers  Per  Day 


Some  Profiles  Resulted  in 
Over  100  Triggers  Per  Day 


Thin 


Mean  =  19.17 
Median  =  14.97 


Sampled  #  of  Triggers  Per  Day 


(U)  Figure  9:  Results  of  Analysis  of  353  Background  Aerosol  Concentration  Profiles  Using  Met-1 
Trigger  Algorithm. 
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(U)  BA  WS  Performance 

(U)  As  with  the  Met-1  trigger  unit  of  the  Portal  Shield,  the  BAWS  trigger  unit  was 
subject  to  widespread  field  use  to  determine  its  performance.  The  BAWS  developers 
report  BAWS  performance  (and  biological  agent  detection  systems  in  general)  as  receiver 
operating  curves  (ROC).  Figure  10  shows  an  example  of  ROC  for  a  few  sets  of  BAWS 
background  data.  These  curves  show  the  trade-off  between  detector  sensitivity  and  false 
positive  rate31. 
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False  Trigger  Rates 
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(U)  Figure  10:  Example  of  ROC  For  Several  Sets  of  BAWS  Background  Data  A 

(U)  The  BAWS  developers  have  a  software  system  called  Testbed  that  allows  the 
developers  to  take  recorded  BAWS  raw  sensor  signals  and  vary  the  detection  algorithm 
parameters  for  optimal  results.  Figure  11  shows  an  example  output  from  Testbed  using 
BAWS  sensor  signals  recorded  during  various  simulant  releases  at  Dugway  Proving 
Grounds.  The  BAWS  developers  can  adjust  how  the  raw  sensor  data  is  compiled  into 
threat  counts  data,  which  represent  the  intensity  of  possible  agent  aerosol  concentrations. 
Testbed  also  allows  the  threshold  trigger  level  to  be  adjusted  so  that  true  triggers,  (i.e., 
when  the  threat  count  for  a  known  release  is  above  the  threshold  level)  are  maximized 
and  false  triggers  are  minimized. 

(U)  The  adjustment  process  was  done  in  two  parts.  The  threat  count  parameters  were 
adjusted  using  data  from  known  simulant  releases  and  agent  signatures  obtained  in 
aerosol  chambers.  The  simulant  release  test  data  included  the  signal  from  the  background 
aerosol.  Whereas,  the  agent  releases  occurred  in  controlled  aerosol  chambers  and  thus, 
the  recorded  data  included  no  outside  background  aerosol  effects.  During  these  tests, 
other  referee  samplers  determined  the  agent  or  simulant  cloud’s  aerosol  concentration, 
which  allowed  the  threat  count  data  to  be  converted  to  agent/simulant  concentration,  as 
measured  in  agent-containing  particles  per  liter  of  air  (ACPLA).  The  threshold  trigger 
level  was  initially  adjusted  using  recorded  data  from  various  BAWS  sampling  locations. 
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Long  term  sampling  at  various  locations  allowed  the  background  signals  to  be  recorded 
and  characterized  so  that  the  number  of  false  triggers  per  day  versus  threshold  trigger 
level  can  be  determined.  The  BAWS  developers  then  combined  the  background  trigger 
level  data  with  the  agent/simulant  data  to  determine  the  ROC  as  shown  in  Figure  10. 
These  data  showed  that  the  sensor’s  performance  was  location  specific  and  that  local 
optimization  required  data  to  be  collected  over  a  long  period  of  time  in  order  to  perform 
local  tuning. 
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(U)  Figure  11:  Example  Output  From  Testbed  Using  BAWS  Sensor  Signals  Recorded  During 
Various  Simulant  Releases  at  Dugway  Proving  Grounds.  Raw  spectral  sensor  values  were  processed 
into  threat  counts.  The  trigger  threshold  was  then  adjusted  to  maximize  threat  triggers  and  minimize  false 
triggers  from  dirt  and  other  unknown  materials33. 

(U)  As  part  of  this  project,  the  BAWS  Testbed  software  system  was  obtained  and  used  to 
analyze  a  few  recorded  BAWS  data  sets.  However,  the  complexity  of  this  system 
precluded  a  large-scale  analysis  of  various  BAWS  location  data  sets,  as  was  done  with 
the  Portal  Shield  Met- 1  data. 

(U)  Since  these  data  were  presented,  the  BAWS  developers  have  continued  to  improve 
the  BAWS  trigger  algorithm.  Data  presented  at  the  DARPA  Special  Projects  Office 
Sensor  Testing  Workshop  on  9  December  200434  indicated  that  the  BAWS  has  a  false 
trigger  rate  between  one  and  10  per  day  at  a  threshold  level  of  100  ACPLA  and  between 
10  to  100  false  triggers  per  day  for  a  threshold  level  of  25  ACPLA35.  This  ROC  data  is  a 
result  of  long  term  (duration)  field-testing  in  Boston,  Missouri,  Atlanta,  Cambridge,  and 
Kuwait,  among  other  locations.  These  data  showed  a  wide  degree  of  variation  from 
location  to  location,  so  these  false  trigger  rates  were  just  estimates  for  an  unknown 
location.  Other  data  collected  from  BAWS  indicated  that  the  device  had  problems 
associated  with  diesel  exhaust  and  particles  associated  with  subway  train  operations. 
Further,  the  more  sophisticated  optics  used  in  this  device  required  more  frequent  cleaning 
(than  a  Met-1  type  device)  in  order  for  the  device  to  operate  at  nominal  levels.  Given  the 
time  over  which  the  BAWS  has  been  developed,  and  the  fact  that  improved  versions  of 
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1  BAWS  have  been  proposed,  this  suggests  that  these  data  represent  the  performance  limits 

2  of  current  technology.  How  well  this  device  can  be  expected  to  perfonn  at  any  untested 

3  location  cannot  be  predicted. 

4 

5  (U)  Continuous  Sample  Collection  Device  Performance 

6  (U)  Testing  performed  by  DARPA  (reference  the  DARPA  Sensor  Testing  Workshop) 

7  showed  that  vegetative  bacterial  cells  cannot  be  cultured  after  drying  out  on  filter  media, 

8  and  that  filter  media  residue  affects  PCR  assays.  Background  material  was  shown  to 

9  reduce  the  sensitivity  of  some  assays,  (e.g.,  HHAs)  by  a  factor  of  ten36.  The  effect  of 

10  background  material  on  PCR  assays  has  not  been  determined.  PCR  analysis  of  material 

11  collected  on  filter  media  resulted  in  a  false  positive  rate  of  about  1%37.  This  false 

12  positive  rate  can  be  reduced  (amount  unspecified)  by  applying  an  independent  secondary 

13  assay  on  other  genetic  loci  within  the  organism.  Multiple  officials  associated  with  the 

14  Biowatch1  program  have  stated  that  millions  of  samples  have  been  processed  without  a 

15  false  positive38.  This  statement  conflicts  with  the  above  false  positive  data  and  with  the 

16  Naval  Research  Laboratory  (NRL)  report  that  found  endemic  F.  tularensis  and  Y.  pestis 

TO 

17  in  one  out  of  every  500  samples  .  A  Biowatch  official  stated  that  endemic  agent 

18  material  is  detected  using  quantitative  PCR  (Q-PCR).  This  technique  relies  on  counting 

19  the  number  of  amplification  cycles  necessary  to  increase  the  target  signal  to  a  detectable 

20  level.  Samples  with  small  amounts  of  initial  segment  material,  (i.e.,  assumed  to  be  small 

21  amounts  of  endemic  background  material)  require  a  larger  number  of  amplification 

22  cycles  than  samples  with  large  amounts  of  initial  segment  material,  which  are  assumed  to 

23  be  collected  from  an  intentional  biological  release.  The  number  of  amplification  cycles 

24  required  for  detection  of  the  sample  can  be  compared  to  the  number  of  cycles  required  for 

25  detection  of  a  positive  control  to  determine  relative  initial  sample  concentration.  This  is  a 

26  controversial  approach  to  filtering  out  endemic  agents  since  the  sample  may  contain  PCR 

27  inhibiting  material.  Although  Q-PCR  is  routinely  used  to  quantify  sample  material  in 

28  laboratories,  this  application  relies  on  well-characterized  samples.  The  reliability  of  Q- 

29  PCR  techniques  on  samples  containing  unknown  background  materials  has  not  been 

30  established.  These  data  suggest  that  the  relationship  between  background  material  and  its 

3 1  effect  on  detection/identification  for  continuous  samplers  is  not  well  known  at  this  time. 

32 

33 

34 


1  Biowatch  is  an  environmental  monitoring  system  developed  by  the  Department  of  Homeland  Security  that 
monitors  air  samples  on  a  frequent  basis  in  major  urban  cities  across  the  U.S. 
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(U)  Conclusions 

(U)  Particle  Monitors/Triggers 

(U)  Simple  particle  counters  designed  to  monitor  the  background  concentration  of 
particles  in  various  size  categories,  such  as  the  Met-1  used  in  the  Portal  Shield,  are 
subject  to  a  large  number  of  trigger  events.  These  trigger  events  are  caused  by  passing 
aerosol  clouds  that  occur  naturally  and  from  human  activities.  Experience  with  simple 
particle  counters  has  shown  that  the  number  of  trigger  events  is  generally  too  high, 
especially  when  used  around  heavy  human  activities  when  the  detection  level  is  set  to  a 
useful  (adequate  sensitivity)  level. 

(U)  Fluorescent  type  particle  counters,  e.g.,  Fluorescence  Aerosol  Particle  Sizer 
(FLAPS),  are  more  sophisticated,  in  that  they  count  particles  that  fluoresce  under  the  idea 
that  most  background  materials  are  inorganic  and  thus  non-fluorescing.  Multiple  studies 
indicated  that  much  of  the  background  material  fluoresces  and  fluorescent  particle 
concentration  does  not  correlate  well  with  agent/simulant  concentration.  A  simple 
FLAPS  device  offered  little  improvement  over  simple  particle  counters. 

(U)  Sophisticated  multi-spectral  monitors  look  at  reflected  light  from  illuminated 
particles  at  multiple  wavelengths  under  the  idea  that  various  types  of  particles  and 
organisms  have  different  spectral  signatures.  The  BAWS  is  an  example  of  this  type  of 
instrument.  Field  results  indicated  that  these  devices  are  still  plagued  by  background 
materials. 

(U)  Particle  monitor  triggers  all  suffer  from  background  concentration  fluctuations, 
which  are  frequent,  natural  and  man-made,  and  unpredictable.  Each  unique  type  of 
particle  monitor  requires  a  field  study  to  determine  how  its  particular  suite  of  sensors  will 
react  to  the  background  materials  at  different  locations.  In  order  to  operate  these  devices 
at  reasonable  sampling  rates,  their  sensitivities  are  adjusted  to  the  point  where  they  are 
able  to  trigger  on  only  the  more  intense  concentration  events.  Their  operational 
sensitivities  are  not  adequate  to  satisfy  operational  requirements  document  (ORD) 
requirements.  It  is  important  to  note  that  the  tenn  “trigger  event”  is  used  rather  than  the 
tenn  “false  positive,”  as  these  devices  initiate  a  sample  collection  and  possible 
identification  sequence.  These  devices  are  designed  to  detect  specific  aerosol 
concentration  signatures  that  might  indicate  the  presence  of  a  BW  agent  cloud.  These 
concentration  signatures  occur  all  too  frequently.  These  are  not  false  alarms. 

(U)  The  effects  of  background  aerosols  on  new  detection  equipment  technologies  cannot 
be  predicted.  Each  new  detection  technology  requires  a  significant  field  study  effort  to 
determine  how  the  new  technology  “sees”  the  environment.  Experience  has  shown  that 
the  environment  is  very  different  for  different  locations  and  sampling  times.  This  means 
that  such  field  study  activities  must  occur  over  many  locations  and  relatively  long  periods 
of  time.  In  cases  where  the  detectors  are  fielded  as  arrays,  and  include  multi-detector 
algorithms,  field  testing  of  a  multi-detector  subset  is  the  only  way  to  determine  the 
system’s  background  response  characteristics.  Existing  data  sets  collected  from 
detectors/samples  of  a  different  design,  cannot  be  used  to  estimate  the  expected 
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performance  characteristics  of  a  different  type  of  system.  This  makes  it 
difficult/expensive  to  perform  the  field  testing  necessary  to  gain  confidence  in  the  level 
of  increased  perfonnance  for  any  new  detection  system. 

(U)  Triggered  Detection  Systems 

(U)  Detection  systems  such  as  Portal  Shield  and  JBPDS  consist  of  arrays  of  detection 
units  that  require  correlated  triggers  (operating  in  Smart  Mode)  before  sample 
identification  procedures  are  initiated.  Positive  agent  identification  of  collected  samples 
must  be  confirmed  by  a  second  identification  assay  before  the  possible  presence  of  agent 
is  declared.  In  such  a  case,  a  sample  is  subject  to  yet  another  independent  identification 
assay  before  an  alarm  signal  is  issued.  Because  of  these  requirements  for  multiple  types 
of  identification  assays,  current  BW  detection  systems  have  very  low  system  level  false 
alarm  rates.  Background  aerosol  materials  contribute  to  increased  rates  of  individual 
sensor  sampling  activities.  Increased  rates  of  sampling,  due  to  background  aerosols  with 
resulting  negative  identification  results,  is  often,  and  incorrectly  referred  to  as  false 
sampling  or  false  triggering.  The  trigger  devices  are  performing  as  they  are  designed. 
The  background  aerosol  material  increases  the  number  of  trigger/sample  events,  which 
usually  leads  to  an  increase  consumption  rate  of  identification  reagents.  If  the  sampling 
rate  and  reagent  consumption  rate  is  deemed  too  high,  then  either  the  system  is  turned  off 
or  the  trigger  thresholds  are  raised.  In  either  case,  the  detection  components  and  total 
system  are  operating  at  a  reduced  sensitivity.  The  system  user  is  provided  no  guidance  as 
to  the  level  of  performance  of  such  a  degraded  system. 

(U)  Continuously  Collecting  Systems 

(U)  Little  technical  data  is  available  relating  the  effects  of  background  aerosol  levels  on 
continuously  collecting  detection  systems.  The  Air  Force  does  not  use  these  systems 
routinely.  Such  systems  have  fixed  reagent  consumption  rates  and  require  transportation 
and  laboratory  support  for  sample  transport  and  processing.  The  Biowatch  program 
utilizes  this  type  of  detection  device,  but  detailed  technical  results  were  not  available  for 
analysis  during  this  effort.  Results  from  this  program  and  the  NRL  effort  suggest  that 
background  material  does  not  contribute  to  false  positive  results.  Results  of  the  DARPA 
effort  and  the  fact  that  Biowatch  does  not  react  to  endemic  agent  material  that  must  be  in 
their  samples  suggests  that  current  processes  are  not  very  sensitive,  although  good  data  to 
support  this  conclusion  is  not  available. 
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(U)  Recommendations 

(U)  After  a  BAWS  has  been  in  the  field  for  some  time,  it  is  possible  to  construct  a  local 
ROC  so  that  false  trigger  rates  could  be  balanced  against  the  threshold  trigger  level  and 
its  relation  to  the  current  threat  condition.  When  BAWS  is  used  with  systems  using 
identification  technologies  of  limited  sensitivity,  such  as  the  JBPDS,  it  is  possible  that  the 
limiting  detection  factor  is  the  identification  stage,  and  not  the  background  induced  false 
trigger  rate.  An  effort  should  be  made  to  investigate  the  possibility  of  generating 
operational  guidance  on  the  relationship  between  BAWS  trigger  threshold  level,  false 
trigger  rate,  and  threat  based  on  local  ROC  data. 

(U)  As  more  sensitive  identification  technologies,  such  as  PCR  come  on-line,  it  will 
become  more  important  to  be  able  to  distinguish  endemic  agent  background  material 
from  deliberate  agent  releases  (attacks).  An  effort  should  be  made  to  collect  the  raw 
assay  data  from  the  Biowatch  program  and  examine  it  to  determine  frequency  and 
characteristics  of  sampled  endemic  agent  material.  These  data  will  be  useful  for  the 
determination  of  the  ROC  data  for  both  triggered  and  continuous  sampler  systems  using 
PCR  identification. 
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(U)  Appendix  A 

(U)  Modeling  Portal  Shield/Met-1  Trigger  Algorithm  to 
Determine  the  Number  of  Trigger  Events  Resulting  from 
Various  Concentration  Time  Series 

(U)  The  Portal  Shield/Met-1  trigger  algorithm  was  obtained  from  (Sentel  Corporation40) 
and  implemented  as  a  computer  code  to  generate  Met-1  trigger  events  in  response  to  an 
input  concentration  time  series,  as  it  would  be  obtained  directly  from  the  Met-1  device. 
The  algorithm  is  complex  and  an  effort  was  made  to  validate  its  implementation  by 
comparing  the  newly  coded  algorithm  results  to  records  of  actual  Portal  Shield  trigger 
responses.  The  data  used  to  analyze  background  aerosol  concentrations  was  reported  by 
Met-1  devices  within  Portal  Shield  units  at  various  times  and  locations.  These  archived 
datasets  also  contained  data  indicating  the  state  of  the  real  trigger  algorithm.  Figure  A1 
shows  the  results  of  one  of  these  validation  tests. 
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Verification  of  Portal  Shield  Detection  Algorithm  -  S1999  (1 1  group,  1  bin  1) 

tsecs  |  Sample  Duratfemple  Statj  Air  Flow  |  PPCC  lannel  1  Colannel  2  Colannel  3  Colannel  4  Colannel  5  Colannel  6  Colqqer  Statulqqer  Statufaqer  Statufaqer  Statujqqer  Statufrqqer  S 


p  e  Model  Implementation  of  Trigger  Algorithm  (bin  1 ) 


(U)  Figure  Al:  Verification  of  Portal  Shield  Detection  Algorithm.  The  top  curve  is  point-to-point 
difference  (PPCC).  The  middle  curve  shows  the  reported  Portal  Shield  trigger  state  from  bin  1  particle 
concentration  data.  The  bottom  curve  is  the  computed  trigger  state  determined  by  the  implementation  of 
the  Portal  Shield  trigger  algorithm. 
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(U)  The  input  concentration  signal  (in  particles  per  cubic  centimeter  from  binl)  is  shown 
as  the  top  curve.  Bin  1  is  one  of  the  Met-1  size  channels.  The  reported  trigger  state  is 
shown  as  the  middle  curve.  Trigger  states  are  as  follows:  1  =  initializing;  2  =  operating 
(green);  3  =  yellow  (possible  trigger  signal  detected);  and  4  =  red  (trigger  signal).  The 
results  of  the  new  code  are  shown  as  the  bottom  curve.  The  reported  trigger  states  in  the 
bottom  curve  are  the  same  as  middle  curve,  but  are  offset  by  one  unit,  (i.e.,  3  =  red  rather 
than  4  =  red).  Examination  of  these  results  shows  that  the  trigger  algorithm 
implementation  is  very  close  to  the  performance  of  the  actual  trigger  algorithm.  There 
are  some  differences  in  de-trigger  timing  (how  the  algorithm  resets)  and  in  the  presence 
of  very  noisy  signals.  Figure  A2  shows  the  results  of  the  trigger  algorithm  validation  for 
another  particle  size  bin.  Again,  good  results  are  obtained.  Qualitative  comparisons  were 
made  for  three  more  sample  profiles  with  similar  levels  of  accuracy.  Based  on  theses 
comparisons,  the  code  was  deemed  sufficient  for  use  in  processing  the  remaining  Met- 1 
type  concentration  profiles  for  use  in  computing  the  number  of  trigger  events  that  can  be 
expected  from  the  wide  range  of  available  background  aerosol  data  sets. 
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(U)  Figure  A2:  Verification  of  Portal  Shield  Detection  Algorithm  Using  a  Second  Particle  Size 
Category.  The  top  curve  is  point-to-point  difference  (PPCC).  The  middle  curve  shows  the  reported  Portal 
Shield  trigger  state  from  bin  1  particle  concentration  data.  The  bottom  curve  is  the  computed  trigger  state 
determined  by  the  implementation  of  the  Portal  Shield  trigger  algorithm. 
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